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Why  an  AoA  Manual? 

Shortcomings  of  Existing  Guidance 


•  Focus  on  AoAs  in  the  pre-JCIDS  environment 

•  Focus  on  quantitative  comparison  of  detailed 
designs,  not  qualitative  comparison  of  conceptual 
designs 

•  Focus  on  platform  and  weapon  systems  rather  than 
Information  Systems  and  processes 

•  Limited  guidance  on  concept  and  capability 
refinement 

•  Limited  guidance  on  specific  tools  and  techniques 
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Acquisition  Framework 

Early  Concept  Decision  &  Refinement 


I  JCIDS 

I  Activities  prior 
|  to  the  Concept 
I  Decision 
I  Point-FAA, 

I  FNA,  FSA,  ICD 


User  Needs  &  Technology 
Opportunities 


Process  entry  at  Milestones  A,  B  or  C 
Entrance  criteria  must  be  met  before 
entering  phase 

Evolutionary  acquisition  or  Single  Step 
to  full  capability 


Concept 

Technology 

System 

Production  & 

Operations 

Refinement 

Development 

Development  & 

Deployment 

&  Support 

AoA 

Demonstration 

Concept 

^Decision  AoA 

TDS 

LRIP  /  x  Design 

IOT&E  Readiness 

V  Review 

Readiness 
V  Review 

f  TDS 

CDD 

Pre-Systems  Acquisition 


Systems  Acquisition 


Sustainment 
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AoAs  in  Spiral  Development 

Not  a  One-Time  Event 
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Milestone 

Decision 


TDS*  ~  Technology  Development  Strategy 
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Information  Systems  (IS) 

Different  from  PlatformA/Veapons 
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ISs  are  primarily  concerned  with  handling 
and/or  processing  of  information 

A  major  goal  of  many  ISs  is  to  increase  the 
value  of  information  by  the  application  of 
appropriate  processes — e.g.,  to  move  from  -< 
signals  to  intelligence 

ISs  can  often  be  modeled  as  a  process 
model--  a  sequence  of  human  and/or 
computer-controlled  steps  in  the  information 
processing  chain 

Compared  to  major  weapon  systems,  ISs 
generally: 

-  Follow  a  rapid  technology  cycle  (1-3  years) 

-  Are  frequently  upgraded ,  even  during  design 

-  Have  shorter  lifetime  before  replacement 


r 

IS  Hierarchy 

Intelligence 

Knowledge 

Information 

Data 

Signals 
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What  Must  the  AoA  Address? 

Six  Specific  Topics 
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•  The  AoA  plan  and  analysis  should  address  the 
specific  topics  highlighted  by  DoD  5000.2  for 
PA&E  assessment  of  the  AoA: 

-  Illuminated  capability  advantages  and  disadvantages 

-  Considered  joint  operational  plans 

-  Examined  sufficient  feasible  alternatives 

-  Discussed  key  assumptions  and  variables  and  sensitivity  to 
changes  in  these 

-  Assessed  technology  risk  and  maturity 

-  Calculated  costs 

•  The  AoA  also  provides  the  basis  for  the  Technology 
Development  Strategy  (TDS)  document 
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AoA  Manual  Outline 

(13  chapters--  ~  175  pages) 


UNCLASSIFIED 


1.  Introduction 

2.  Defense  System  Acquisition  Framework  (JCIDS  Overview) 

3.  AoA  Methodology 

4.  AoA  Study  Plan 

5.  Preparing  for  Analysis 

6.  AoA  Analysis  Strategy 

7.  Capabilities  Refinement 

8.  Concept  Refinement 

9.  Performance  Analysis 

10.  Cost  Analysis 

1 1 .  Technology  Risk  Analysis 

12.  Performance-Cost-Risk  Tradeoffs 

13.  Final  Results 

Six  Appendices  (including  examples  of  analysis  techniques) 
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AoA  Methodology  (Chapter  3) 

An  Iterative  Process 


UNCLASSIFIED 


•Key  data  or  tools  are 
unavailable;  what  is  a 
viable  alternative 
analysis  approach? 

•  The  size  of  the 
scenario  space  is  not 
clear;  what  is  the  full 
operating  range  that 
is  required? 


•  The  analysis  tools 
are  inadequate;  are 
there  suitable 
alternative  tools? 

•  Can’t  calculate  the 
required 
parameters;  are 
there  equivalent 
alternative 
parameters? 


•  Concepts  don’t  meet 
the  performance 
criteria  and  concept 
refinements  don’t  solve 
the  problem;  is  a  new 
conceptual  approach 
needed? 

•  The  analysis  results 
don’t  identify  the  best 
designs;  are  we 
calculating  the  correct 
parameters? 
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AoA  Team  Structure  (Chapter  5) 

No  Universal  Style 


"Conventional"  AoA  Team  Structure 


Candidate  Team  Structure  for  Initial  AoA  in  JQDS 
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Analysis  Strategy  (Chapter  6) 

How  to  Move  between  Tasks 


UNCLASSIFIED 


Capabilities  are: 


capabilities 


filter  to 

eliminate  non- 
viable  designs) 


*  P-C-R:  Performance-Cost-Risk 
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Candidate  Tools  and  Techniques 

Where  They  Work 


Tools  & 
Techniques 

Concept 

Refinement 

Capability 

Refinement 

P-C-R 

Calculations 

Trade-off 

Analysis 

Subject 

Matter 

Experts 

X 

X 

X 

X 

ATAM 

X 

P,R 

Analysis  by 
Analogy 

p,  C,  R 

QAM 

X 

P,R 

M&S 

P,R 

Hierarchical 

Structures 

X 

X 

QFD 

X 

X 

TRIZ 

X 
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Candidate  Tools  and  Techniques 

Where  They  Work  (cont.) 


Tools  & 

Concept 

Capability 

P-C-R 

Trade-off 

Techniques 

Refinement 

Refinement 

Calculations 

Analysis 

MADT-  VFT 

X 

MADT-AHP 

X 

Technology 
Readiness 
Level  (TRL) 

R 

Technology 

S-curves 

X 

R 

Fast 

Heuristics 

X 

Parametric 

Costing 

C 

Engineering 

Costing 

C 

Cost 

Sufficiency 

C 
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Summary 
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•  AoAs  have  changed  significantly  under  JCIDS  with  a 
broader  range  of  roles  and  activities  for  the  AoA  team: 

-  Capability  refinement 

-  Concept  refinement 

-  Performance-Cost-Technology  Risk  Assessments 

-  Trade-off  analysis 

•  AoAs  before  Milestone  A  frequently  may  not  have 
quantitative  metrics 

•  The  AoA  is  not  a  one-time  event;  it  is  reviewed  and  revised 
or  redone  in  spiral  development 

•  A  number  of  analysis  tools  are  available  for  the  AoA  that 
work  well  with  qualitative  metrics 

•  The  new  Manual  provides  specific  guidance  for  conducting 
AoAs  in  this  new  environment 
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Contact  Information 
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Kevan  L.  Barton 

National  Security  Agency 

9800  Savage  Rd 

Ft.  George  G.  Meade,  MD 
20755 

443-479-5816 


Dr.  Charles  H.  Sinex 

Johns  Hopkins  University 
Applied  Physics  Laboratory 

11100  Johns  Hopkins  Road 
Laurel,  MD  20723-6099 
240-228-5617 
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Back-up  slides 
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Issues  with  DoD  Acquisition 

Why  JCIDS? 
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•  Need  to  plan  for  future  uncertainty 

-  JCIDS  adopts  Capability  Based  Planning 

•  Very  long  product  development  times 

-  JCIDS  adopts  Evolutionary  Acquisition  w/  Spiral  Development 
as  the  preferred  strategy 

•  Stove-piped  planning 

-  JCIDS  is  top-down  driven  from  a  Joint  perspective 

•  Emphasis  on  process  rather  than  product 

-  JCIDS  promotes  flexibility  subject  to  compliance  with 
statutory  and  regulatory  requirements 

•  Difficulty  in  introducing  “novel,  innovative”  systems 

-  AoA  must  consider  alternative ,  novel  concepts  and  help  define 
a  pre-acquisition  decision  Technology  Development  effort 
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The  Defining  JCIDS  Documents 

A  Sequence  of  Four 


- ► 

In  accordance^ 

with'" 

p 

In  accordance 

with 

_ 

In  suppoi 

rt  of 

5000.2 

Instruction 


Implements 


Authorizing  document 


1  Establishes  the  JCIDS  1 


framework  and  AoA 


I 


Defines  3170  series 
as  the  process 


i  focus 


3170.01E 

JCIDS 

Instruction 


J  Establishes  JCIDS 
policies  and  procedures 


I 


3170.01B 
Operation  of  JCIDS 
Manual 


Establishes  policy  &  procedures; 
references  31 70.01  B  for  specific 
procedures 

r 


i  Establishes  guidelines  and 

- 1  procedures  for  development  and 

1  staffing  of  JCIDS  documents 


Sets  forth  guidelines  &  procedures 
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Capabilities  Based  Planning 

What  is  it  in  JCIDS? 
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•  Capabilities  definition  (3170) 

-  “The  ability  to  execute  a  specified  course  of  action.  It  is  defined  by  an 
operational  user  and  expressed  in  broad  operational  terms  in  the 
format  of  an  initial  capabilities  document  or  a  DOTMLPF 
recommendation.  In  the  case  of  material  proposals ,  the  definition  will 
progressively  evolve  to  DOTMLPF  performance  attributes  identified  in 
the  CDD  and  the  CPD” 

•  Attributes  definition  (3170) 

-  “A  testable  or  measurable  characteristic  that  describes  an  aspect  of  a 
system  or  capability ” 

•  Metrics  definition  (based  on  general  community  usage) 

-  Quantifiable  parameters  that  show  to  what  extent  attributes  are 
achieved  by  a  particular  system 

-  Metrics  have  three  elements 

•  A  clear  description  of  the  parameter  being  measured 

•  A  testable  and  measurable  value  for  its  threshold 

•  A  testable  and  measurable  value  for  its  objective 
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Where  They  are  Specified 
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Generic  Hierarchical  Task 

Structure 
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All  tasks  are  required  (logical  AND) 

Any  combination  of  tasks  is  acceptable  (logical  OR) 
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Hierarchical  Structure  Example 

Concept  or  Capability  Refinement 


Md  nito  r  TV  Trans  mis  s  s  io n 
for  Quality 


“Refining”  capabilities 
inevitably  narrows  the 
solution  space  and 
eventually  leads  to 
specific  tasks 


1 

i 

i 

Receive 

Analyze 

Modify 

Signal 

Signal 

Transmission 

i 

. i _ 

@  Transmitter 

Md-path 

@  Receiver 

Process 

Signal 

Looking 
upwards- 
activities  look 
like  capabilities 

Looking 
downwards— 
activities  look 
like  tasks 


Locate  & 
Point  Antenna 


Link  to 
Receiver 


Set  Receiver 
Frequency 


Provide 

Power 


Interpret 

Signal 


Demodulate 

Signal 


Demultiplex 

Signal 


Filter 

Signal 


Record 

Data 


Post- 

Real- 

Transmission 

lime 

Frequency 

Stability 


Amplitude 

Stability 


Determine 

Modifications 


Provide 

Recommendations 


Audio 

Clarity 


Video 

Quality 


All  tasks  are  required  (AND) 

Any  combination  of  tasks  is  acceptable  (OR) 
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Architecture  Framework 

Required  Description  for  New  Systems 
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Notional  OV-5  Activity  Diagram 

Key  View  for  IT  System  Analysis 
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